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Abstract 


It is well known that nonlinear distortion over a communication channel is now a significant 
factor hindering further increase in the attainable data rate in high-speed data transmission. Since the 
received signal over a nonlinear channel is a nonlinear function of the past values of the transmitted data 
pulses, it is not surprising that the linear equalizers do not work efficiently. We propose a new nonlinear 
equalizer that uses a new type of neural network called Fuzzy CMAC which combines the advantages of 
both fuzzy logic and CMAC (Cerebellar Model Arithmetic Computer) networks. The learning speed is 
an order of magnitude faster than conventional neural nets. Moreover, human expert knowledge in the 
form of linguistic rules can be easily incorporated into the scheme. 


1. Introduction 


In practical digital communications systems designed to transmit at high speed over band-limited 
channels, a succession of pulses transmitted through the channel at rates comparable to the channel 
bandwidth are smeared to the point that they are no longer distinguishable as well-defined pulses at the 
receiving terminal [ |]. This is due to several factors. First, linear amplitude and delay distortion caused 
by the nonideal channel frequency response characteristic makes the symbols overlap. Also the delay 
distortion makes the zero crossings no longer periodically spaced. Second, signals transmitted through 
telephone channels are subject to other impairments such as nonlinear distortion, frequency offset, phase 
jitter, impulse noise, and thermal noise. Nonlinear distortion in telephone channels is due to the 
nonlinearities in amplifiers and compandors used in the telephone system. This type of distortion is very 
difficult to correct. A small frequency offset (< 5 Hz) results from the use of carrier equipment in the 
telephone channel. Such an offset cannot be tolerated in high-speed digital transmission systems which 
use synchronous phase-coherent demodulation. The offset can be compensated for by the carrier 
recovery loop in the demodulator. Phase jitter is a low--index frequency modiulation of the transmitted 
signal with the low frequency harmonics of the power line frequency (50 or 60 Hz). Phase jitter poses a 
serious problem in digital transmission of high rates. However, it can be tracked and compensated for at 
the demodulator. /mpu/se noise is an additive disturbance. It comes from the switching equipment in the 
telephone system. 7hermal noise is also present at levels of 20 to 30 dB below the signal. 
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In this paper, we will deal with the signal distortion due to the linear and nonlinear channel 
characteristics and additive noises. Since frequency offset and phase jitter can be remedied by some 
other means, we will not discuss these impairments. 

The popular approach to channel equalization is to model the channel as a linear finite-impulse- 
response (FIR) filter. Then an adaptive filter is used to cancel the distortion caused the channel. Various 
weight adjustment schemes such as LMS (Least Mean Square), RLS (recursive least-squares), and lattice 
algorithms are used to tune the filter parameters [1] [2]. However, these schemes can only handle linear 
distortions. It is well known that nonlinear distortion over a communication channel is now a significant 
factor hindering further increase in the attainable data rate in high-speed data transmission. Since the 
received signal over a nonlinear channel is a nonlinear function of the past values of the transmitted data 
pulses, it is not surprising that the linear equalizers do not work efficiently. Therefore, an efficient and 
effective equalizer should posses certain capability that can learn the nonlinear behavior of the channel 
characteristics. In [4] and [6], polynomial adaptive filters were developed for nonlinear channel 
equalization. In [5], multilayer perceptrons have been used as equalizers. In [3], fuzzy adaptive filters 
were used to cancel the nonlinear distortion. 

Since nonlinear channels cover many kinds of nonlinear distortions, it is hard to single out one 
method that is dominant. Hence it is necessary to develop new nonlinear equalizers and this is the goal 
of this paper. We propose a new nonlinear equalizer that uses a new type of neural network called Fuzzy 
CMAC which combines the advantages of both fuzzy logic and CMAC (Cerebellar Model Arithmetic 
Computer) networks. The learning speed is an order of magnitude faster than conventional neural nets. 
Moreover, human expert knowledge in the form of linguistic rules can be easily incorporated into the 
scheme. 

The paper is organized as follows. In section 2, we will introduce some backgrounds on Fuzzy 
CMAC. Then we will describe the channel equalization problem and our approach in section 3. 
Simulation results will be included. Finally, conclusions will be drawn in Section 4. 


2. Fuzzy CMAC Neural Network 


2.1 Comparison of a CMAC Network with a Fuzzy Logic Controller (FLC) 


Two decades ago, a unique neural network model called a Cerebellar Model Arithmetic 
Computer (CMAC) was established by J. Albus [8] based on a model of the human memory and 
neuromuscular control system. The CMAC is a perceptron-like associative memory that performs 
nonlinear function mapping over a particular region of a function space. A CMAC network has the 
capability to learn an unknown nonlinear mapping by examples, and to reproduce multiple outputs in 
response to multiple inputs. Because of its table look-up mechanism, and its hash-code based mapping 
structure, CMACs are able to cope with high dimensional input/output applications without severely 
deteriorating their processing speed and performance. 

Fuzzy set theory was initially proposed by L. Zadeh as a tool to model the imprecision that is 
inherent in human reasoning, especially when dealing with complexity. The fuzzy theory has seen its 
most widespread application in the area of control. Controllers using control laws specified with fuzzy 
set theory (or fuzzy logic) are known as fuzzy controllers. Such controllers are easier (relative to non- 
fuzzy controllers) to design, especially in cases where the laws are non-linear and the systems are 
complex. 
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A fuzzy logic controller consists of an input tuzzification module, a set of fuzzy control rules 
based on which the approximate reasoning technique is used to make a control decision, and an output 
defuzzification module. 

Fuzzy logic controllers differ from classical math-model controllers in that they do not require an 
explicit mathematical model of how control outputs functionally depend on control inputs. Also fuzzy 
logic controllers allow designers to incorporate human knowledge into the control decision making 
process. 

The advantages of a CMAC over a FLC are: 

° There are very efficient learning laws to update the values of weights based on experience and 
examples. 

e There is a random mapping mechanism to reduce the physical memory requirement for multiple 
input and high resolution applications. 

e There exist efficient input encoding schemes for high dimensional input vectors. 

The advantages of a FLC over a CMAC are: 

e It is possible to interpret the implication of weight values using linguistic labels. 

e The membership functions and the firing strengths contain additional information as to how close 
the input vector is to each linguistic variable. Therefore the number of input space partitions may 
be smaller to achieve the same generalization and output smoothness. 

® The fuzzy rules can take a variety of forms while only numeric values can be associated with 
CMAC associative memory locations. 

e There are many methods to construct a fuzzy control knowledge base, such as expert’s experience 
and knowledge. 

In the next section. we will propose a new neural network called Fuzzy CMAC which combines 


the advantages of both fuzzy logic and CMAC network. 


2.2 Fuzzy CMAC Architecture 


In this section, we present the novel concept of a Fuzzy CMAC neural network. This novel 
network was developed by Intelligent Automation Inc. (IAI). The idea is to combine the advantages of 
the FLC and the CMAC (Cerebellar Model Arithmetic Computer) and eliminate the respective 
disadvantages of the two. IAI has successfully applied this network to active vibration control, finger 
print analysis, and chaotic time-series prediction. Fig. 1 illustrates the architecture of the Fuzzy CMAC. 
The Fuzzy CMAC inherits the preferred features of arbitrary function approximation, learning, and 
parallel processing from the original CMAC neural network, and the capability of acquiring and 
incorporating human knowledge into a system and the capability of processing information based on 
fuzzy inference rules from fuzzy logic. The combination of a neural network and fuzzy logic vields an 
advanced intelligent system architecture. 

At the input stage, the Fuzzy CMAC uses the fuzzification method of a FLC as its input 
encoding scheme. Fuzzy rules can be assigned to each associative memory cell. These rules may not 
necessarily have a crisp consequent part. The output generation uses a defuzzification approach which 
sums weighted outputs of the activated rules based on the firing strengths s;. The overall mapping 


function of a Fuzzy CMAC can be formalized as 


M 
Z(u = > Say, (2.1) 
p=i 
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where u = [u, 7 ee uy] is the input vector. w,, p =1, 2,.., M, are the weights of the network. 
N iv] 

A4=j,if N= 1 and M= dG, -~D)[ [m, + j,,forN>1.i=1,2,...,M. mj; is the number of knot 
j=2 II 


points on the it input dimension. The j;" knot point on the ith input dimension is denoted as Uns 
J; = 1,2,...,m,.5,,p=1,2,..., M, are the firing strength of the fuzzy rules. They are calculated as 


Si > FE i, (us, )* jy (U, JFL (uy) (2.2) 
where * denotes the T-norm operator. There are many types of T-norms such as the min and product 
operators. Throughout this paper, we choose the T-norm as the product inference method since it is easy 
to implement. Hence, (2.2) can be written as 


N 
Sp= Hl ial (u, Vain (u, ). wf, iy (uy) = I] a i (u;) (2.3) 
where yu ; (u;) is the j/" membership function of the i* input. 

A Fuzzy CMAC neural network combines the advantages listed for CMAC and FLC. One 
desirable feature the Fuzzy CMAC inherits from the CMAC model is that the receptive field of the 
sensing element has limited width. This means that there are only a small number of sensing elements to 
‘be activated for any sensor reading. In conventional neural nets such as those based on multiple layer 
feedforward structure and backpropagation learning algorithms, all neurons are required to perform 
computation in order to compute the forward mapping or to perform learning. In Fuzzy CMACs, 
operations are localized and only a small subset of all the neurons need to be computed. Our initial 
comparisons of fuzzy CMACs with conventional neural nets show orders of magnitude increase in the 
speed of both function mapping and learning for typical problems on conventional computational 
hardware. Comparing the activating function of CMAC to the linguistic variables in the Fuzzy CMAC, 
one can view the activating function as the membership function of the Fuzzy CMAC input variables. 
Fig. 2 shows the comparison. 

In terms of the control knowledge rule-base, the proposed Fuzzy CMAC differs from Albus’s 
CMAC [8] in that the weight values can be interpreted as knowledge rules through linguistic variables. 
This feature permits us to validate a learned Fuzzy CMAC in terms of the reasonableness of the learning 
results. This unique feature also provides a practical channel for knowledge acquisition. A knowledge 
base (a set of rules) can be established based on the learning results of a Fuzzy CMAC. 

On the other hand, the Fuzzy CMAC distinguishes itself from Zadeh’s fuzzy controller in that it 
is able to build a learning control system starting with an empty knowledge base. The linguistic 
knowledge of human experts can be incorporated into these rules. Based on the initial knowledge base, a 
self learning algorithm is employed to modify the existing rules in order to improve the system 
performance. 


2.3 Fuzzy CMAC Mapping Using B-Spline Membership Function 


The fuzzy membership function can be chosen with considerable freedom, and this freedom can 
be used to optimize system performance. Membership functions should be computationally simple, 
flexible, and continuous in order to optimize the Fuzzy CMAC system design. We have compared many 
candidates for membership function, such as bell shape Gaussian functions, sinusoidal functions, etc., 
and we believe that the B-Spline function family possesses certain preferred properties that make them 
well suitable for the Fuzzy CMAC membership function. Let the j;* knot point on the i" input dimension 
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be denoted as I, ,. “Throughout this paper. we use the third order B-spline function. The firing strength 


of a fuzzy membership function /(u.) is the value of the function , at the input u . For example, if u 
lies in the range off My, (se, ; (w)]. then the firing strength of the ;;" membership function will be 


Li Ui; =i en Uj ae ae u— uy | 
5, = | Za eee se (2.3) 
- Ue, —U,,_, U;, —U u ue. , ue, —U 


ri, bey = lef, ij-l if, +1 _ i ace ea ijn] 


The third order B-Spline membership functions have the following desirable properties: 


(a) Positivity: 4, (u;) > O for all uw, elu, ; 34 a 
(b) Compact support: “/, , (u;) = 0 for all u, not belonging to uy, du, salt, | 
(c) Normalization: » di, ,(u;)=I1 for any u, . 


(d) Derivatives exist and can be recursively calculated. 


It should be noted that the positivity property is not that important in Fuzzy CMAC. However, the other 
three properties require more attention. The existence of derivatives is very important for many real-time 
control applications where the tracking errors are back-propagated through the Fuzzy CMAC. Compact 
support property means that? for a given input vector, only a small number of the membership functions 
will be fired and need to be computed. For our third order B-spline function, only 3 membership 
functions will be activated in each input dimension. Thus only a small number of weights need to be 
updated. In other words, our Fuzzy CMAC enjoys the localization property which represents a 
considerable amount of’ computational savings when compared to a general feedforward 
backpropagation neural network which must calculate all the network weights whenever there is a new 
input vector. The smoothness of the B-spline functions also give the Fuzzy CMAC the generalization 
property which means similar inputs will give similar outputs. The normalization property will help in 
simplifying the computations. Since the output of a Fuzzy CMAC can be expressed as 


My MI y my 


be ae SIT] 0 inte sf al 


z(u) = em (ele os ee (2.5) 


My My ny 2 


> LTA 


lie =! jp=li= 


it seems from the above formula that the an of the numerator as well as the denominator of the 


z(u) both involve a summation of M (= ie ) terms, each in turn requires N multiplications. By using 
i=l 
the normalization property of the B-spline membership function, the calculation of the z/1) becomes 


much simpler. This fact is proven in the following theorem. 


Theorem 1 
Because of the normalization property of B-spline function, the denominator in (2.1) always 


takes the value I, e.g. 


My Wy | My 


pe cel (2.6) 


fy =lyy = J =lr=l 


Proof: See [7]. 
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2.4 Fuzzy CMAC Universal Approximation Theorem 

We have proven the following universal approximation theorem for the Fuzzy CMAC neural 
network. The theorem essentially guarantees that the Fuzzy CMAC neural network can uniformly 
approximate any nonlinear continuous function over a specific compact region to any degree of 
accuracy. The universal approximation theorem provides us with justification for applying the Fuzzy 
CMAC to almost any nonlinear system modeling problem. 


Theorem 2 
For any given real continuous function g(u) on a compact set U c R" , and arbitrary ¢> 0, there 
exists a Fuzzy CMAC neural network z(u) in the form of (2.1) such that 
sup|z(u) — 9(u)|<é for VucU OF 
where u = [u, Uy, 6 uy] is the input vector; w(j,, j,,.... jy) are the weights of the Fuzzy CMAC; 
and yz, , are the membership function values. 
Proof: See [7]. 


2.5 Learning Algorithm for the Fuzzy CMAC 


The conventional CMAC learning process is designed so that the correction initiated by an 
output error is evenly distributed among all weights that contribute to the output, regardless of their true 
proportion of contribution to the output. In the Fuzzy CMAC, a more intelligent method is adopted in 
which the correction of the output error is distributed to the weights in proportion to their contribution. 
The weight updating algorithm is described below. 

Given a training pair (u, zg) where u=[u,, U2,..., Ux]! is input vector and Zg is the desired 
output, update weights and knot points associated with fired fuzzy rules, such that the error between z 
and zg is reduced. Suppose there is only one output and many inputs. The Fuzzy CMAC mapping can 
be expressed in terms of the product of firing strength and weights 


M 
z=) s,w,. (2.8) 
p=i 


The learning algorithm is targeted to reduce error between the desired output Zg and the actual output of 
Fuzzy CMAC by adjusting the values of weights 


l 2 1 2 
H=—(z,-z) =~e. (2.9) 
2 (21-2) 2 
The adjustment of the weights is based on the following rules 
Aw, = ~fes,, p= 1, 2,...,M (2.10) 


where § is the learning rate. These learning rules differ from the original CMAC learning rule proposed 
by Albus in that the adjustment of a weight is proportional to its contribution (measured by its firing 
strength) to the output signal based on which the error is generated. The conventional CMAC distributes 
the output error evenly among all the contributing weights. 


3. Nonlinear Channel Equalization Using Fuzzy CMAC 


A typical digital communication system is shown in Fig. 3 [3]. The channel includes the effects 
of the transmitter filter, the transmission medium, the receiver matched filter, and other components. The 
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transmitted data sequence s( k ) is assumed to be an ‘ndependent sequence taking values from} - 1.1! 
with equal probability. The inputs to the equalizer. x(k). x(A —1),...- x(k -- 7 + 1) are the channel 
outputs corrupted by an additive noise 7(4). The task of the equalizer at the sampling instant k is to 
produce an estimate of the transmitted symbol s(&4 —¢) using the information contained in y( 4 ), 
x(kA~-1),...,x(4-n-+4+ 1). The integers n, ¢@ are known as the order and lag of the equalizer. 


respectively. 
The equalization problem can be formulated as follows. Similar to [5], we define 


P, q (1) = 48(k) ER" s(k -d) = Hh, 


P,g(-l) = (2k) €R"|s(k —d) = -1} (3.1) 
where 

P , F a 

£(k) =[X(4), SK -D MAA th]. 
Note that x(x) is the noise-free output of the channel, and P, ,, (1) and P, ,; (-1) represent the two sets 
of possible channel noise-free output vectors x(4) that can be produced from sequences of the channel 
inputs containing s( k — d) = I and s(k — d) = -1, respectively. The equalizer can be characterized bv 
the function 

g, RR! j-1 I} (3.2) 
with 

s(k — d) = gy (x(k) 
where 

x(k) =[x(k).x(k = 1). ek nt Db]! 
is the observed channel output. Let p, | x(4|8(4) ely (| and p_, [ x(k) k) € P, 4 (-1)| be the 
conditional probability density functions of x(k) given x(k) ¢ P, g (1) and x(ALeE P, ag (-1). 
respectively It was shown in [5] that the equalizer which is defined by 

Foy (3) = sent py (WEA) € Pg (D) = PAR EP, gD 8.3) 


achieves the minimum bit error rate for the given order # and lag qd, where 
sen(v) = | (-1) i/v 2 0 (v <0) . If the noise q(k) is zero-mean and Gaussian with covariance matrix 


O = Elk) ek — nt Dk), mk + D)"T, (3.4) 
then 
Py(x(k) x(k) € P,Q) = py (x()(k) € Pg (-D) 
| . ae . 
= exp] — (at), Oath -8,)] - > exp) Fh) £97 0 al -£_). 
€€P,y() : &ePy(-b 2 


Let us consider a nonlinear channel described by the following discrete model 
<(k) = S(k) + 0.5s(k — 1) — 0.9[s(k) + 0.58(k - DP (3.5) 
and the white Gaussian noise 7(k) with FE (7° (k)) = 0.2. The optimal decision boundary for this case is 
shown in Fig. 4. The elements of the sets P; , (1) and P, , (-1) are illustrated in Fig. 4 by the “o” and 


‘““k® respectively. It can be seen that the boundary is very nonlinear. Linear equalization techniques will 
not perform well for such a nonlinear boundary. 
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Here we propose to use Fuzzy CMAC as a nonlinear equalizer. The procedure consists of two 
steps. 


Step 1: Generation Of training data 


For Fuzzy CMAC to be an efficient nonlinear equalizer, a thorough training of various possible 
cases is necessary. From Fig. 4, it can be seen there are 8 possible noise free elements. We generate 
10,000 data pairs around the 8 noise free elements by adding Gaussian noise to them. The data sets are 
shown in Fig. 5. The desired outputs of the network are either | or -]. The Fuzzy CMAC equalizer has 
two inputs and one output. The training of the network is done by evaluating the error and the error is 
used to adjust the weights in the Fuzzy CMAC. We used 10 membership functions for each input of the 
equalizer. 


Step 2: Performance test 


To test the performance of the nonlinear equalizer, we generate another 100,000 data pairs with 
various noise amplitudes. The output of the Fuzzy CMAC is compared with the desired values to check 
whether the decision is right or wrong. Due to the generalization property of the Fuzzy CMAC, the 
network can make good decision about untrained cases. The overall performance is shown in Fig. 6, 
which shows the bit error rate versus the signal-to-noise ratio. 


4. Conclusion 


A new nonlinear equalizer using Fuzzy CMAC neural network is proposed to perform 
equalization for nonlinear channels. The training speed of the network is an order of magnitude faster. 
Moreover, the human expert knowledge can be incorporated into the equalizer design. Simulation results 
show that the performance is similar to existing approaches. 


References 


[1]J. Proakis, Digital Communications, McGraw-Hill Book Co., New York, 1989. 

[2] C.F.N. Cowan, “Performance Comparisons of Finite Linear Adaptive Filters,” JEE Proc., vol. 134, 
pt. F, no.3, pp. 211-215, 1987. 

[3] L. Wang, Adaptive Fuzzy Systems and Control, Prentice-Hall, New Jersey, 1994 

[4] E. Biglieri, A. Gersho, R.D. Gitlin, and T.L. Lim, “Adaptive cancellation of nonlinear intersymbol 
interference for voiceband data transmission,” JEEE J. on Selected Areas in Communications, no.5, 
pp. 765-777, 1984. 

[ 5] S. Chen, G.J. Gibson, C.F.N. Cowan, and P.M. Grant, “Adaptive equalization of finite nonlinear 
channels using multilayer perceptrons,” Signal Processing, vol. 20, pp. 107- 119, 1990. 

[6] D.D. Falconer, “Adaptive equalization of channel nonlinearities in QAM data transmission systems,” 
The Bell System Technical Journal, vol. 57, no. 7, pp. 2589-2611, 1978. 

[7] Z. Geng, H. Xu, and L.S. Haynes, “Fuzzy CMAC Neural Network Application to Flight Vehicle 
Control,” Phase I report to U.S. Army, November, 1994. 

[8] J. Albus, “A new approach to manipulator control: The cerebellar model articulation controller 
(CMAC),” Trans. ASME, Journal of Dynamic Systems, Measurement and Control, vol. 97, pp. 220- 
221,1975. 


32 


Fuzzy Fire Associated 


uated Strength Fuzzy Memory 
Rasoning Cell Vector 
Rules ' 
Sai 


ian 
i 
\ 


é wa 
Cc 
p< 
C— Cee: 
‘Ss Weighted Sum 
= re 
u Cy 
Input = 
Space Ss 
eS Weight Adjustrent 
—> 
u 
é Green > e 
B 7 + 
* Learning Rule ze 
d 


WwieWt f(z -z) SS 
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Fig. 6 Bit error rate versus signal to noise ratio. 
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